Image sensor, photographing method, and image-capturing device

ABSTRACT

An image sensor includes: a plurality of microlenses arranged in a two-dimensional pattern; and a plurality of pixels that are provided in correspondence to each of the microlenses and receive lights of different color components, respectively. Pixels that are provided at adjacent microlenses among the microlenses and that receive lights of same color components, are adjacently arranged.

TECHNICAL FIELD

The present invention relates to an image sensor, a photographing methodand an image-capturing device.

BACKGROUND ART

An image-capturing device is known which performs focus detection by asplit-pupil phase detection method on the basis of output signals from aplurality of pixels dedicated for focus detection arranged on a part ofan image sensor (see Patent literature 1).

CITATION LIST Patent Literature

-   Patent literature 1: Japanese Patent Application Laid-open No.    2007-282109.

SUMMARY OF INVENTION Technical Problem

To perform focus detection by the conventional art, the focus detectionis possible only at positions where pixels dedicated for focus detectionare arranged. However, an increased number of pixels dedicated for focusdetection results in a decrease in image quality since no image signalis obtained from the positions where the pixels dedicated for focusdetection are arranged. As described above, according to theconventional technology, it is made possible to perform focus detectionby a phase detection method as well as generation of image signals basedupon output signals from image sensor. On the other hand, there occursan adverse effect due to provision of the pixels dedicated for focusdetection at a part of the image sensor.

Solution to Problem

According to the 1st aspect of the present invention, an image sensorcomprises: a plurality of microlenses arranged in a two-dimensionalpattern; and a plurality of pixels that are provided in correspondenceto each of the microlenses and receive lights of different colorcomponents, respectively. Pixels that are provided at adjacentmicrolenses among the microlenses and that receive lights of same colorcomponents, are adjacently arranged.

According to the 2nd aspect of the present invention, in the imagesensor according to the 1st aspect, it is preferred that signals fromthe pixels are used both for focus detection and for generation of imagedata.

According to the 3rd aspect of the present invention, an image sensorcomprises: a plurality of microlenses arranged in a two-dimensionalpattern; and a plurality of pixels that are arranged in correspondenceto each of the microlenses and receive lights having different colorcomponents, respectively. Signals from the pixels are used both forfocus detection and for generation of image data.

According to the 4th aspect of the present invention, a photographingmethod comprises: a focus detection step for performing focus detection,by using signals from particular pixels in an image sensor including aplurality of microlenses arranged in a two-dimensional pattern and aplurality of pixels that are arranged in correspondence to each of themicrolenses and receive lights having different color components, theparticular pixels receiving lights of same color components fromdifferent microlenses, respectively; and a shooting step for generatinga capture image by using signals from plurality of pixels provided incorrespondence to each of the microlenses, the plurality of pixelsreceiving lights of different color components.

According to the 5th aspect of the present invention, in thephotographing method according to the 4th aspect, it is preferred that,in the image sensor, pixels are provided at adjacent microlenses amongthe microlenses and that receive lights of same color components areadjacently arranged.

According to the 6th aspect of the present invention, an image-capturingdevice comprises: an image sensor that captures an image of a subjectwith light fluxes from the subject that have passed through an imagingoptical system; an image generation unit that generates an image signalbased upon an output signal from the image sensor; and a focus detectionunit that detects a focusing condition of the imaging optical system bya phase detection method based upon an output signal from the imagesensor. The image sensor includes a pixel group and a microlens grouparranged so as to guide the light fluxes from the subject to the pixelgroup. The pixel group includes first, second and third pixels havingfirst, second and third spectral sensitivities, respectively, differingfrom each other, and being arranged in a two-dimensional pattern, withone of the first pixels, one of the second pixels and two of the thirdpixels being arranged in a two-by-two matrix behind each microlens inthe microlens group, and the four pixels receive four light fluxes,respectively, that pass through four pupil areas, respectively, of anexit pupil of the imaging optical system. The first, second, and thirdpixels are arranged such that pixels having substantially same spectralsensitivities are adjacently arranged in a two-by-two matrix and fourpixels adjacent to the two-by-two matrix are arranged behind fourdifferent microlenses in the microlens group, respectively, and atdifferent positions with respect to the microlenses. The imagegeneration unit generates the image signal based upon output signalsfrom the first, second and third pixels. The focus detection unitdetects the focusing condition based upon an output signal from at leastone of the first, second, and third pixels.

According to the 7th aspect of the present invention, in theimage-capturing device according to the 6th aspect, it is preferred thatthe first pixel has a red color filter, the second pixel has a bluecolor filter, and the third pixel has a green color filter.

According to the 8th aspect of the present invention, in theimage-capturing device according to the 7th aspect, it is preferred thatthe pixel group includes an array of a plurality of sets of pixelsarranged in a two-dimensional pattern, each of the plurality of sets ofpixels having four pixels arranged in a two-by-two matrix behind anyparticular one of the microlenses and the sets include first throughfourth sets having different arrangements of pixels, in the first set,the first pixel and the third pixel are adjacently arranged in apredetermined array direction and the third pixel and the second pixelare arranged adjacent to the first pixel and the third pixel,respectively, in a direction perpendicular to the predetermined arraydirection, in the second set, the third pixel and the first pixel areadjacently arranged in the predetermined array direction and the secondpixel and the third pixel are arranged adjacent to the third pixel andthe first pixel, respectively, in the direction perpendicular to thepredetermined array direction, in the third set, the third pixel and thesecond pixel are adjacently arranged in the predetermined arraydirection and the first pixel and the third pixel are arranged adjacentto the third pixel and the second pixel, respectively, in the directionperpendicular to the predetermined array direction, in the fourth set,the second pixel and the third pixel are adjacently arranged in thepredetermined array direction and the third pixel and the first pixelare arranged adjacent to the second pixel and the third pixel,respectively, in the direction perpendicular to the predetermined arraydirection, the first set and the second set are adjacent to each otherin the predetermined array direction and alternately arranged in arepeated manner in the predetermined array direction, the third set andthe fourth set are adjacent to each other in the predetermined arraydirection and alternately arranged in a repeated manner in thepredetermined array direction, a first row formed by the first set andthe second set and a second row formed by the third set and the fourthset are adjacent to each other in the direction perpendicular to thepredetermined array direction and alternately arranged in a repeatedmanner in the direction perpendicular to the predetermined arraydirection.

According to the 9th aspect of the present invention, in theimage-capturing device according to the 7th or the 8th aspect, it ispreferred that the image generation unit adds output signals from fourof the first pixels that are adjacent to each other in a form of atwo-by-two matrix, adds output signals from four of the second pixelsthat are adjacent to each other in a form of a two-by-two matrix, andadds output signals from four of the third pixels that are adjacent toeach other in a form of a two-by-two matrix to generate an image signalof a Bayer arrangement.

According to the 10th aspect of the present invention, in theimage-capturing device according to any one of the 6th through 8thaspects, it is preferred that the image generation unit obtains threecolor signals at a position corresponding to each microlens based uponoutput signals from the first, second and third pixels arranged behindeach microlens.

According to the 11th aspect of the present invention, in theimage-capturing device according to any one of the 6th through 8thaspects, it is preferred that the image generation unit performs, atrespective positions of the first through third pixels, colorinterpolation processing for generating signals of other two spectralcomponents to obtain three color signals and generates a luminancesignal and color difference signals based on the thus obtained threecolor signals.

According to the 12th aspect of the present invention, in theimage-capturing device according to any one of the 6th through 11thaspects, it is preferred that the focus detection unit detects thefocusing condition of the imaging optical system based upon outputsignals from a pair of pixels having substantially same spectralsensitivities and located at positions differing from each other withrespect to the microlens, out of the pixel group.

According to the 13th aspect of the present invention, in theimage-capturing device according to the 8th aspect, it is preferred thatthe focus detection unit detects the focusing condition of the imagingoptical system in the predetermined array direction based upon outputsignals from at least one plurality of the third pixels of a pluralityof the third pixels contained in the first set and the second set,respectively, and a plurality of the third pixels contained in the thirdset and the fourth set, respectively.

According to the 14th aspect of the present invention, in theimage-capturing device according to the 8th aspect, it is preferred thatthe focus detection unit detects the focusing condition of the imagingoptical system in the direction perpendicular to the predetermined arraydirection based upon output signals from at least one plurality of thethird pixels of a plurality of the third pixels contained in the firstset and the third set, respectively, and a plurality of the third pixelscontained in the second set and the fourth set, respectively.

According to the 15th aspect of the present invention, in theimage-capturing device according to the 8th aspect, it is preferred thatthe focus detection unit detects the focusing condition of the imagingoptical system in a direction oblique to the predetermined arraydirection based upon output signals from at least one plurality of thethird pixels of a plurality of the third pixels contained in the firstset and the fourth set, respectively, and a plurality of the thirdpixels contained in the second set and the third set, respectively.

According to the 16th aspect of the present invention, an image sensorcomprises: a pixel group; and a microlens group arranged so as to guidethe subject light fluxes to the pixel group. The pixel group includesfirst, second and third pixels having first, second and third spectralsensitivities, respectively, differing from each other, and beingarranged in a two-dimensional pattern, with one of the first pixels, oneof the second pixels and two of the third pixels being arranged in atwo-by-two matrix behind each microlens in the microlens group, and thefour pixels receive four light fluxes, respectively, that pass throughfour pupil areas, respectively, of an exit pupil of an imaging opticalsystem. The first, second, and third pixels are arranged such thatpixels having substantially same spectral sensitivities are adjacentlyarranged in a two-by-two matrix, respectively, and four pixels adjacentto the two-by-two matrix are arranged behind four different microlensesof the microlens group, respectively, and at different positions withrespect to the microlenses.

Advantageous Effect of Invention

According to the present invention, image signal generation and focusdetection by a phase detection method can be performed based upon outputsignals from the image sensor without providing the image sensor withpixels dedicated for focus detection.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 A diagram illustrating by an example a configuration of a digitalcamera according to an embodiment of the present invention.

FIG. 2 A plan view illustrating by an example a layout of pixels in animage sensor.

FIG. 3 A diagram illustrating by an example an exit pupil of aninterchangeable lens.

FIG. 4 A diagram illustrating pixel rows for obtaining a defocus amount.

FIG. 5 A diagram illustrating light fluxes passing through an exitpupil.

FIG. 6 A diagram illustrating pixel rows for obtaining a defocus amount.

FIG. 7 A diagram illustrating light fluxes passing through an exitpupil.

FIG. 8 A diagram illustrating pixel rows for obtaining a defocus amount.

FIG. 9 A diagram illustrating light fluxes passing through an exitpupil.

FIG. 10 A diagram illustrating first image signal generation processing.

FIG. 11 A diagram illustrating second image signal generationprocessing.

FIG. 12 A diagram illustrating third image signal generation processing.

FIG. 13 A diagram illustrating third image signal generation processing.

FIG. 14 A diagram illustrating third image signal generation processing.

FIG. 15 A diagram illustrating third image signal generation processing.

FIG. 16 A diagram illustrating third image signal generation processing.

FIG. 17 A flowchart illustrating the flow of imaging processing.

FIG. 18 A diagram illustrating image signal generation processingaccording to Variation Example 6.

FIG. 19 A diagram illustrating by an example a circuit configuration ofan image sensor.

FIG. 20 A plan view illustrating by an example a layout of circuitry inan image sensor.

FIG. 21 (a) A diagram illustrating by an example an incidence plane ofan image sensor, (b) a diagram illustrating by an example a wiring planeof an image sensor.

FIG. 22 A diagram illustrating by an example connection between an imagesensor and a signal processing chip.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the present invention is described withreference to the attached drawings. FIG. 1 presents a diagramillustrating by an example the configuration of a digital cameraaccording to an embodiment of the present invention. A digital camera 1includes an interchangeable lens 2 and a camera body 3. Theinterchangeable lens 2 is mounted on the camera body 3 via a mount unit4.

The interchangeable lens 2 includes a lens control unit 5, a main lens9, a zoom lens 8, a focusing lens 7, and a diaphragm 6. The lens controlunit 5 includes a microcomputer, a memory and so on and performs drivecontrol of the focusing lens 7 and the diaphragm 6, detection of an openstate of the diaphragm 6, detection of positions of the zoom lens 8 andthe focusing lens 7, transmission of information about lens to a bodycontrol unit 14 on the side of the camera body 3 described later,reception of information about camera from the body control unit 14, andso on.

The camera body 3 includes an image sensor 12, an image sensor drivecontrol unit 19, the body control unit 14, a liquid crystal displayelement drive circuit 15, a liquid crystal display element 16, an ocularlens 17, an operating member 18 and so on. A detachable memory card 20is attached to the camera body 3. The image sensor 12 is arranged on apredetermined imaging plane of the interchangeable lens 2 and capturesan image of a photographic subject that is formed by the interchangeablelens 2.

The body control unit 14 includes a microcomputer, a memory and so on.The body control unit 14 controls operations of the digital camera inwhole. The body control unit 14 and the lens control unit 5 areconfigured to communicate with each other via an electric contact unit13 of the mount unit 4.

The image sensor drive control unit 19 generates a control signal thatis necessary for the image sensor 12 in response to a command from thebody control unit 14. The liquid crystal display element drive circuit15 drives the liquid crystal display element 16 that constitutes aliquid crystal view finder (EVF: electric view finder) in response to acommand from the body control unit 14. The photographer observes animage displayed on the liquid crystal display element 16 through theocular lens 17. The memory card 20 is a storage medium in which imagedata and the like are stored.

The image of the photographic subject formed in the image sensor 12 bythe interchangeable lens 2 is subjected to photoelectric conversion bythe image sensor 12. The image sensor 12 is controlled by a controlsignal from the image sensor drive control unit 19 with respect totiming (frame rate) of storage of photoelectric conversion signals andreading of signals. The output signal from the image sensor 12 isconverted in an A/D conversion unit (not shown) into digital data andthe obtained digital data is transmitted to the body control unit 14.

The body control unit 14 calculates a defocus amount on the basis ofoutput signals from the image sensor 12 corresponding to predeterminedfocus detection areas and transmits the calculated defocus amount to thelens control unit 5. The lens control unit 5 calculates a focusing lensdrive amount on the basis of the defocus amount received from the bodycontrol unit 14 and drives the focusing lens 7 by a motor or the like,which is not shown, to move it to a focusing position on the basis ofthe calculated lens drive amount.

The body control unit 14 generates image data to be recorded on thebasis of a signal that is output from the image sensor 12 after shootingis commanded. The body control unit 14 stores the generated image datain the memory card 20 and at the same time transmits the generated imagedata to the liquid crystal display element drive circuit 15 and controlsit to be reproduced and displayed on the liquid crystal display element16.

It is to be noted that the camera body 3 is provided with the operatingmember 18 that includes a shutter button and a setting member forsetting a focus detection area or areas and so on. The body control unit14 detects an operating signal from the operating member 18 and controlsthe operations (photographing processing, setting of a focus detectionarea and so on) in response to the result of the detection.

<Description of Image Sensor>

Since the present embodiment is featured by the image sensor 12, furtherdescription is focused on the image sensor 12. FIGS. 2( a) and (b)presents plan views each illustrate by an example the layout of pixelsin the image sensor 12. In this case, 10×10 pixels, which are extractedfrom the pixels in the image sensor 12 as representatives, are shown.The extracted pixels are laid out in a substantially square form andarranged in a two-dimensional pattern. The extracted pixels includethree types of pixels, i.e., a pixel that receives light of a red colorcomponent (R pixel), a pixel that receives light of a green colorcomponent (G pixel), and a pixel that receives light of a blue colorcomponent (B pixel).

The R pixel includes a color filter that is transmissive to only thelight of the red color component and a photoelectric conversion unitarranged behind this color filter. The G pixel includes a color filterthat is transmissive to only the light of the green color component anda photoelectric conversion unit arranged behind this color filter. The Bpixel is constituted by a color filter that is transmissive to only thelight of the blue color component and a photoelectric conversion unitarranged behind this color filter.

In addition, the image sensor 12 is formed of a plurality of microlenses40 each of which efficiently guides light fluxes from theinterchangeable lens 2 to a set of four pixels. In FIG. 2, 5×5=25circles correspond to the microlenses 40. The microlenses 40, eachconstituted by a spherical lens of axial symmetry whose centersubstantially coincides with its optical axis or a nonspherical lens,are arranged in a two-dimensional pattern, with the light incident sidethereof having a convex shape.

Behind each of the microlenses 40, one R pixel, two G pixels and one Bpixel are arranged in a two-by-two matrix. In the present invention, asshown in FIG. 2( a), a plurality of sets of four pixels positionedbehind the corresponding microlenses 40, respectively, are classifiedinto four types (P1 through P4) according to differences in theirarrangements.

Behind the microlenses 40, first sets P1 each include an R pixel at aleft upper position, a G pixel at a right upper position, a G pixel at aleft lower position, and a B pixel at a right lower position. Secondsets P2 each include a G pixel at a left upper position, an R pixel at aright upper position, a B pixel at a left lower position, and a G pixelat a right lower position. Third sets P3 each include a G pixel at aleft upper position, a B pixel at a right upper position, an R pixel ata left lower position, and a G pixel at a right lower position. Fourthsets P4 each include a B pixel at a left upper position, a G pixel at aright upper position, a G pixel at a left lower position, and an R pixelat a right lower position.

The first sets P1 and the second sets P2 are adjacent to each other in ahorizontal direction (X direction) and arranged alternately in arepeated manner in the horizontal direction. A line formed by the firstsets P1 and the second sets P2 is called a “first line L1”. The thirdsets P3 and the fourth sets P4 are adjacent to each other in thehorizontal direction and are arranged alternately in a repeated mannerin the horizontal direction. A line formed by the third sets P3 and thefourth sets P4 is called a “second line L2”.

The first line L1 and the second line L2 described above are adjacent toeach other in a vertical direction (Y direction) and are alternatelyarranged in a repeated manner in the vertical direction. With thisconfiguration, each of the first sets P1 and each of the third sets P3are adjacent to each other in the vertical direction, whereas each ofthe second sets P2 and each of the fourth sets P4 are adjacent to eachother in the vertical direction.

With such an arrangement, the microlenses 40 and the R pixels, the Gpixels, and the B pixels have the following positional relationships.

First, the R pixels, behind four microlenses 40 adjacent to each otherin the horizontal direction and in the vertical direction, are arrangedat a left upper position, a right upper position, a left lower position,and a right lower position, respectively. The G pixels, behind fourmicrolenses 40 adjacent to each other in the horizontal direction and inthe vertical direction, are arranged at right upper and left lowerpositions, left upper and right lower positions, left upper and rightlower positions, and right upper and left lower positions, respectively.The B pixels, behind four microlenses 40 adjacent to each other in thehorizontal direction and in the vertical direction, are arranged at aright lower position, a left lower position, a right upper position, anda left upper position, respectively. In this manner, the R pixels, the Gpixels, and the B pixels are uniformly arranged behind the microlenses40 without being arranged disproportionately to specific positions.

FIG. 2( b) is a diagram showing an extracted part similar to that shownin FIG. 2( a). When four sets of pixels (P1 through P4) shown in FIG. 2(a) are viewed by shifting them by 1 pixel both in the horizontaldirection and in the vertical direction, the R, G and B pixels each arearranged such that adjacent four pixels in a two-by-two matrix have thesame color as shown in FIG. 2( b).

In addition, the four pixels in a two-by-two matrix having the samecolor are arranged behind different microlenses 40, respectively, sothat they assume different positions with respect to the microlenses 40.In other words, the R, G and B pixels arranged behind the microlenses40, respectively, are arranged such that they are adjacent to each otherin a two-by-two matrix for each color.

The sets constituted by four pixels in a two-by-two matrix of the samecolor, i.e., a set 50 r constituted by four R pixels, a set 50 gconstituted by four G pixels, and a set 50 b constituted by four Bpixels, when the four pixels are viewed as one set, each form a Bayerarrangement.

<Focus Detection Processing>

Next, an example in which signals for focus detection are obtained fromthe image sensor 12 having the configuration described above isdescribed referring to FIG. 3 through FIG. 9. FIG. 3 presents a diagramillustrating by an example an exit pupil 80 of the interchangeable lens2 in a state in which the diaphragm is open. Light fluxes that havepassed through four regions 81 through 84 of the exit pupil 80 enterpixels located at positions corresponding to a left upper part, a rightupper part, a left lower part, and a right lower part, respectively, ofeach of the microlenses 40 in FIG. 2. For each of the microlenses 40,correspondence relationship between the light fluxes that enter thepixels located at positions corresponding to the left upper, rightupper, left lower, and right lower parts of the microlens and the firstregion 81, the second region 82, the third region 83, and the fourthregion 84, respectively, of the exit pupil 80 may be considered suchthat the up and down relation as well as the left and right relation areinverted with respect to the light axis Ax of the interchangeable lens 2as an axis of symmetry. It is to be noted that FIG. 3 presents a diagramillustrating a concept. Actually, a configuration is adopted such thatthe light that has passed through each of the microlenses 40 isprevented from entering pixels included in an adjacent set so that therewill occur no color contamination or no decrease in resolution and thelike.

First, as exemplified in FIG. 4, explanation is made on a case in whichthe defocus amount is obtained based on a pixel row 90, in which Gpixels out of the pixels in the image sensor 12 are arranged in thehorizontal direction (X axis direction). The pixel row 90 is constitutedby a G pixel included in the second set P2 and located at the left upperposition of the microlens 40 (G-a) and a G pixel included in the firstset P1 and located at the right upper position of the microlens 40(G-b). As exemplified in FIG. 5, a light flux A that passes through afirst region 81 on the exit pupil 80 and a light flux B that passesthrough a second region 82 on the exit pupil 80 enter the pixels thatconstitute the pixel row 90. The light flux A enters the G pixel (G-a)located at the left upper position of the microlens 40. The light flux Benters the G pixel (G-b) located at the right upper position of themicrolens 40.

Upon focusing, the image sensor 12 is in a state in which a sharp imageis formed, so that as described above, a pair of images formed by lightfluxes through different positions of the pupil as a result ofpupil-splitting coincide with each other on the image sensor 12. Inother words, in the pixel row 90, a signal waveform (signal sequence a1,a2, a3, a4, . . . ) obtained from the G pixels (G-a) that receive thelight fluxes A and a signal waveform (signal sequence b1, b2, b3, b4, .. . ) obtained from the G pixels (G-b) that receive the light fluxes Boverlap in their shape.

On the other hand, upon non-focusing, i.e., in a state in which a sharpimage is formed on the front side or on the rear side of the imagesensor 12, a pair of images formed by the light fluxes subjected to thepupil-splitting do not overlap with each other on the image sensor 12.In this case, the signal waveform (signal sequence a1, a2, a3, a4 . . .) by the light fluxes A and the signal waveform (signal sequence b1, b2,b3, b4, . . . ) by the light fluxes B have different positionalrelationships (deviation direction and deviation amount) therebetweenaccording to a deviation (a defocus amount) from the focused state.

The body control unit 14 calculates the focusing condition (defocusamount) of the focus position by the interchangeable lens 2 on the basisof the positional relationship between the signal waveform (signalsequence a1, a2, a3, a4 . . . ) by the light fluxes A and the signalwaveform (signal sequence b1, b2, b3, b4, . . . ) by the light fluxes Band transmits the result of calculation that serves as camerainformation to the lens control unit 5. As the lens control unit 5 movesthe focusing lens 7 back and forth along the optical axis direction onthe basis of the camera information, the focus is adjusted so that asharp image can be formed on the image sensor 12.

Next, explanation is made on a case in which the defocus amount isobtained based upon a pixel row 120 in which G pixels out of the pixelsin the image sensor 12 are arranged in the vertical direction (Y axisdirection) as exemplified in FIG. 6. The pixel row 120 is constituted bya G pixel (G-a) that is included by the second set P2 and located at aposition corresponding to the left upper part of the correspondingmicrolens 40 and a G pixel (G-c) that is included by the fourth set P4and located at a position corresponding to the left lower part of thecorresponding microlens 40. As shown in FIG. 7, the light flux A thatpasses through the first region 81 on the exit pupil 80 and a light fluxC that passes through a third region 83 on the exit pupil 80 enter thepixels that constitute the pixel row 120. The light flux A enters the Gpixel (G-a) located at a position corresponding to the left upper partof the corresponding microlens 40. The light flux C enters the G pixel(G-c) located at a position corresponding to the left lower part of thecorresponding microlens 40.

Upon focusing, the image sensor 12 is in a state in which a sharp imageis formed therein, so that in the pixel row 120 as described above, asignal waveform (signal sequence a1, a2, a3, a4, . . . ) obtained fromthe G pixels (G-a) that receive the light fluxes A and a signal waveform(signal sequence c1, c2, c3, c4, . . . ) obtained from the G pixels(G-c) that receive the light fluxes C overlap in their shape.

On the other hand, upon non-focusing, the signal waveform (signalsequence a1, a2, a3, a4 . . . ) by the light fluxes A and the signalwaveform (signal sequence c1, c2, c3, c4, . . . ) by the light fluxes Chave positional relationships (deviation direction and deviation amount)therebetween which are different from each other according to adeviation (a defocus amount) from the focused state.

The body control unit 14 calculates the focusing condition (defocusamount) at the focus position by the interchangeable lens 2 on the basisof the positional relationships between the signal waveform (signalsequence a1, a2, a3, a4 . . . ) by the light fluxes A and the signalwaveform (signal sequence c1, c2, c3, c4, . . . ) by the light fluxes Cand transmits the result of calculation that serves as camerainformation to the lens control unit 5. As the lens control unit 5 movesthe focusing lens 7 back and forth along the optical axis direction onthe basis of the camera information, the focus is adjusted so that asharp image can be formed on the image sensor 12.

In addition, explanation is made on a case in which the defocus amountis obtained on the basis of a pixel row 150 that is constituted by Gpixels out of the pixels in the image sensor 12 are arranged in anoblique direction as exemplified in FIG. 8. The pixel row 150 isconstituted by a G pixel (G-a) included in the second set P2 and locatedat a position corresponding to the left upper part of the correspondingmicrolens 40, a pixel (G-d) included in the second set P2 and located ata position corresponding to the right lower part of the correspondingmicrolens 40, a G pixel (G-a) included in the third set P3 and locatedat a position corresponding to the left upper part of the correspondingmicrolens 40, and a G pixel (G-d) included in the third set P3 andlocated at a position corresponding to the right lower part of thecorresponding microlens 40. As shown in FIG. 9, the light flux A thatpasses through the first region 81 on the exit pupil 80 and a light fluxD that passes through a fourth region 84 on the exit pupil 80 enter thepixels that constitute the pixel row 150. The light flux A enters the Gpixel (G-a) located at a position corresponding to the left upper partof the corresponding microlens 40. The light flux D enters the G pixel(G-d) located at a position corresponding to the right lower part of thecorresponding microlens 40.

Upon focusing, the image sensor 12 is in a state in which a sharp imagecan be formed therein, so that in the pixel row 150 as described, asignal waveform (signal sequence a1, a2, a3, a4, . . . ) obtained fromthe G pixels (G-a) that receive the light fluxes A and a signal waveform(signal sequence c1, c2, c3, c4, . . . ) obtained from the G pixels(G-c) that receive the light fluxes C overlap in their shape.

On the other hand, upon non-focusing, the signal waveform (signalsequence a1, a2, a3, a4 . . . ) by the light fluxes A and the signalwaveform (signal sequence d1, d2, d3, d4, . . . ) by the light fluxes Dhave positional relationship (deviation direction and deviation amount)therebetween, which are different from each other, according to thedeviation (defocus amount) from the focused state.

The body control unit 14 calculates the focusing condition (defocusamount) of the focus position by the interchangeable lens 2 on the basisof the positional relationship between the signal waveform (signalsequence a1, a2, a3, a4 . . . ) by the light fluxes A and the signalwaveform (signal sequence d1, d2, d3, d4, . . . ) by the light fluxes Dand transmits the result of calculation that serves as camerainformation to the lens control unit 5. As the lens control unit 5 movesthe focusing lens 7 back and forth along the optical axis direction onthe basis of the camera information, the focus is adjusted so that asharp image can be formed on the image sensor 12.

<Image Signal Generation Processing>

Next, explanation is made on an example in which image signals areobtained from the image sensor 12 referring to FIG. 10 through FIG. 16.In the present embodiment, any of the following three methods is used asimage signal generation processing for generating a color image signalon the basis of output signals from the image sensor 12. The bodycontrol unit 14 executes image signal generation processing by a methodindicated by an initial setting in advance.

(First Image Signal Generation Processing)

FIG. 10 presents a diagram illustrating first image signal generationprocessing. The body control unit 14 that executes the first imagesignal generation processing treats four pixels that receive lightfluxes through the one and the same microlens 40 as one set 200 as shownin FIG. 10( a). Each set 200 includes two G pixels, one B pixel and oneR pixel.

The body control unit 14 treats, for each set 200, an output signal fromthe R pixel as an R image signal of the set 200, an output signal fromthe B pixel as a B image signal of the set 200, and an average value ofoutput signals from the two G pixels as a G image signal of the set 200.As a result, the body control unit 14 can obtain color image signals(RGB) in a number that is ¼ times the number of the pixels included bythe image sensor 12 as shown in FIG. 10( b). The body control unit 14generates an image file for recording by using the thus-obtained colorimage signals.

As described above, in the first image signal generation processing,color image signals can be obtained without performing colorinterpolation processing for interpolating color signals.

(Second Image Signal Generation Processing)

FIG. 11 presents a diagram illustrating second image signal generationprocessing. The body control unit 14 that executes the second imagesignal generation processing treats adjacent four pixels in a two-by-twomatrix having the same color as one set 210 as shown in FIG. 11( a).

The body control unit 14 treats a signal obtained by adding outputsignals from the four pixels included in any particular one of the set120 as an image signal of the particular set 210. Specifically, in thecase of any particular one set 210 all constituted by R pixels, the bodycontrol unit 14 treats a signal obtained by adding output signals fromthe four R pixels as an R image signal of the particular one set 210. Inthe case of any particular one set 210 all constituted by G pixels, thebody control unit 14 treats a signal obtained by adding output signalsfrom the four G pixels as a G image signal of the particular one set210. In the case of any particular one set 210 all constituted by Bpixels, the body control unit 14 treats a signal obtained by addingoutput signals from the four B pixels as a B image signal of theparticular one set 210. As a result, the body control unit 14 can obtaincolor image signals of a Bayer arrangement in a number that is ¼ timesthe number of the pixels included by the image sensor 12 as shown inFIG. 11( b).

And now, depending on the angle of incidence of the light fluxes thatenter the microlens 40 it may happen that the four pixels arrangedbehind the microlens 40 receive uneven amounts of light. For instance,at a certain incident angle θ1, the amount of light received by thepixel located at a position corresponding to the left upper part of thecorresponding microlens 40 is relatively large while the amount of lightreceived by the pixel located at a position corresponding to the rightlower part of the corresponding microlens 40 is relatively small. Atanother incident angle θ2, the amount of light received by the pixellocated at a position corresponding to the left upper part of thecorresponding microlens 40 is relatively small while the amount of lightreceived by the pixel located at a position corresponding to the rightlower part of the corresponding microlens 40 is relatively large.

In the second image signal generation processing, as a signal obtainedby adding output signals from four pixels located at positionscorresponding to different parts (left upper, right upper, left lower,and right lower parts) of the corresponding microlens 40 (that is, fourpixels included by any particular one set 210) is treated as an imagesignal of the particular one set 210, an optimal image signal can begenerated independently of the incident angle of the light fluxes thatenter the microlens 40.

In addition, the body control unit 14 generates, in an image signal of aBayer arrangement in any particular one set 210, a color component thatis in short by interpolation processing using signals from sets 210adjacent to the particular one set 210. For instance, in the case of anyparticular one set 210 all constituted by G pixels, as there is presentneither R image signal nor 13 image signal therefrom, colorinterpolation processing is executed by using signals from sets 210circumjacent to the particular one set 210. Since such colorinterpolation processing in the Bayer arrangement is known, detaileddescription thereof is omitted herein. The body control unit 14generates a file for recording by using color image signals (RGB)obtained by executing this color interpolation processing.

(Third Image Signal Generation Processing)

The body control unit 14 that executes third signal generationprocessing first executes color interpolation processing forinterpolating a color component that is in short in each pixel.

FIG. 12 presents a diagram illustrating processing for interpolating a Gimage signal. The body control unit 14 generates, at a position of anyparticular one of the R pixels and the B pixels, a G image signal byusing output signals from four G pixels located nearest to theparticular one pixel by interpolation processing. For instance, in casethat a G image signal is to be interpolated at the position of the Rpixel in a thick-frame in FIG. 12( a), output signals from four G pixels(G1 through G4) that are located nearest to this particular R pixel areused. The body control unit 14 defines (αG1+βG2+γG3+δG4)/4 as a G imagesignal of the particular R pixel. It is to be noted that α through δeach are coefficients according to the distances from the particular Rpixel. The smaller the distance from the particular R pixel, the greaterthe coefficient is. In this case, as the G pixels G1 and G2 are closerto the particular R pixel than the G pixels G3 and G4 are, it is setthat α=β>γ=δ.

In this manner, the body control unit 14 executes processing forinterpolating G image signals at positions of R pixels and B pixels, sothat a G image signal can be obtained at a position of each pixel 30 asshown in FIG. 12( b).

FIG. 13 presents a diagram illustrating processing for interpolating Rimage signals and B image signals. The body control unit 14 treats fourpixels constituted by adjacent four pixels in a two-by-two matrix havingthe same color as one set 220 and a signal obtained by adding outputsignals from the four pixels is defined to be an image signal of the set220 as shown in FIG. 13( a). The sets 220 each form a Bayer arrangementas shown in FIG. 13( b). The body control unit 14 executes interpolationprocessing for interpolating an R image signal and a B image signal ineach of the sets 220 by using conventional color interpolationprocessing for Bayer arrangements. As a result of this interpolationprocessing, an R image signal is obtained in each set 220 as shown inFIG. 14( a) and a B image signal is obtained in each set 220 as shown inFIG. 15( a).

The body control unit 14 converts resolution by defining a signalobtained by dividing the interpolated image signal in each set 220 by 4(R/4) as an R image signal of the four pixels that constitute each set220. As a result, the body control unit 14 can obtain an R image signalat a position of each pixel 30 as shown in FIG. 14( b). Similarly, thebody control unit 14 converts resolution by defining a signal obtainedby dividing the interpolated image signal in each set by 4 (B/4) as a Bimage signal of the four pixels that constitute each set 220. As aresult, the body control unit 14 can obtain a B image signal at aposition of each pixel 30 as shown in FIG. 15( b).

The body control unit 14 executes the color interpolation processing asdescribed above to obtain an image signal of RGB at a position of eachpixel as shown in FIG. 16( a). The body control unit 14 obtains aluminance (brightness) signal Y at a position of each pixel 30 by usingthe image signal of RGB at a position of each pixel 30. For instance,the body control unit 14 defines 0.299R+0.587G+0.114B as the luminancesignal Y.

In addition, the body control unit 14 defines a signal (R-Y) obtained bydeducing a luminance signal Y from an R image signal at a position ofeach pixel 30 as a color difference (chrominance) signal Cr. The bodycontrol unit 14 defines a signal (B-Y) obtained by deducing a luminancesignal Y from a B image signal at a position of each pixel 30 as a colordifference (chrominance) signal Cb.

As a result, the body control unit 14 can obtain a luminance signal Yand color difference signals Cr and Cb at a position of each pixel 30 asshown in FIG. 16( b). The body control unit 14 generates an image filefor recording by using the color image signals (YCrCb) thus obtained.

<Shooting Processing>

FIG. 17 presents a flowchart illustrating the flow of imaging processingto be executed by the body control unit 14. The body control unit 14,when a main switch (not shown) that constitutes the operating member 18is turned ON, controls the image sensor 12 to start photoelectricconversion at a predetermined frame rate to control the liquid crystaldisplay element 16 to successively reproduce and display a through imagebased on the image signal and at the same starts up a program forexecuting the processing exemplified in FIG. 17. It is to be noted thatthe through image is an image for monitoring that is obtained beforeshooting is commanded.

In step S11 in FIG. 17, the body control unit 14 makes a decision as towhether or not a command for shooting is issued. When a release buttonthat constitutes the operating member 18 is pushed down, the bodycontrol unit 14 makes a positive decision in step S11 and the programproceeds to step S12. When the release button is not pushed down, thebody control unit 14 makes a negative decision in step S11 and theprogram proceeds to step S18.

In step S18, the body control unit 14 makes a decision as to whether ornot time is up. When the body control unit 14 measures a predeterminedtime (for instance, 5 seconds), it makes a positive decision in step S18and the processing in FIG. 17 is terminated. When the measured time isless than the predetermined time, the body control unit 14 makes anegative decision in step S18 and the program returns to step S11.

In step S12, the body control unit 14 executes AE processing and AFprocessing. In the AE processing, exposure is calculated on the basis ofthe level of image signal for the through image and aperture value AVand shutter speed TV are decided so that optimal exposure can beobtained. In the AF processing, the focus detection processing describedabove is executed based on an output signal sequence from the pixel rowincluded in the set focus detection area. When the body control unit 14completed execution of the AE and AF processing described above, theprogram proceeds to step S13.

In step S13, the body control unit 14 executes shooting processing andthe program proceeds to step S14. Specifically, the body control unit 14controls the diaphragm 6 based on the AV and controls the image sensor12 to perform photoelectric conversion for recording for a storage timeon the basis of the TV. In step S14, the body control unit 14 executesthe image signal generation processing by using output signals from theimage sensor 12 and the obtained image signal is subjected topredetermined image processing (gradation processing, contourenhancement, white balance adjustment processing and so on). When thebody control unit 14 has executed the image processing, the programproceeds to step S15.

In step S15, the body control unit 14 controls the liquid crystaldisplay element 16 to display the captured image thereon and the programproceeds to step S16. In step S16, the body control unit 14 generates animage file for recording and the program proceeds to step S17. In stepS17, the body control unit 14 records the generated image file in amemory card 20 and terminates the processing in FIG. 17.

According to the embodiment described above, the following operationsand advantageous effects can be obtained.

(1) As the digital camera 1 includes the image sensor 12 that capturesan image of a subject by a light flux from a subject that passes throughthe interchangeable lens 2; the body control unit 14 that generates animage signal on the basis of output signals from the image sensor 12;and the body control unit 14 detecting a focusing condition of theinterchangeable lens 2 by a phase detection method. The image sensor 12has a pixel group and a microlens group arranged such that the lightfluxes from the subject are guided to the pixel group; the pixel groupincludes an R pixel, a B pixel, and a G pixel having first, second, andthird spectral sensitivities, respectively, arranged in atwo-dimensional pattern; behind each microlens 40, there are arrangedone R pixel, one B pixel and two G pixels in a two-by-two matrix, thesefour pixels each receive four light fluxes A through D, respectively,that pass four pupil regions 81 through 84, respectively, of the exitpupil 80. Further, the R pixel, the B pixel, and the G pixel arearranged such that pixels having substantially the same spectralsensitivities (that is, pixels of the same color) are adjacentlyarranged in a two-by-two matrix, whereas four pixels that are adjacentto the two-by-two matrix are arranged behind four different microlenses40, respectively, such that their positions with respect to themicrolenses 40 are different from each other. The body control unit 14generates an image signal on the basis of output signals from the Rpixel, the B pixel, and the G pixel and the body control unit 14 detectsthe focusing condition on the basis of an output from the G pixel. Withthis configuration, the generation of an image signal on the basis of anoutput signal from the image sensor 12 and the focus detection by thephase detection method can be performed without providing the imagesensor 12 with any pixels dedicated for focus detection.(2) In the digital camera 1 in (1) above, as a configuration is adoptedsuch that the R pixels each have a red color filter, the B pixels eachhave a blue color filter, and the G pixels each have a green colorfilter, red green blue color image signals can be obtained from theoutput signals from the image sensor 12.(3) In the digital camera 1 in (2) above, the pixel group is formed byarranging, in a two-dimensional pattern, a plurality of sets of pixels,each of which set includes four pixels arrayed in a two-by-two matrixbehind one microlens 40, the sets include first through fourth sets inwhich arrangements pixels are different from each other. In the firstset P1, an R pixel and a G pixel are arrayed horizontally adjacent toeach other and a G pixel and a B pixel are arrayed adjacent to thehorizontally arrayed R and G pixels, respectively, in the verticaldirection. In the second set P2, a G pixel and an R pixel are arrayedhorizontally adjacent to each other and a B pixel and a G pixel arearrayed adjacent to the horizontally arrayed G and R pixels,respectively, in the vertical direction. In the third set P3, a G pixeland a B pixel are arrayed horizontally adjacent and an R pixel and a Gpixel are arrayed adjacent to the horizontally arrayed G and B pixels,respectively, in the vertical direction. In the fourth set P4, a B pixeland a G pixel are arrayed horizontally adjacent and a G pixel and an Rpixel are arrayed adjacent to the horizontally arrayed G and B pixels,respectively, in the vertical direction. The first set P1 and the secondset P2 are horizontally adjacent and alternately arrayed in a repeatedmanner in the horizontal direction. The third set P3 and the fourth setP4 are horizontally adjacent and alternately arrayed in a repeatedmanner in the horizontal direction. A first line L1 formed by the firstset P1 and the second set P2 and a second line L2 formed by the thirdset P3 and the fourth set P4 are adjacent to each other in the verticaldirection and alternately arrayed in a repeated manner in the verticaldirection. With this configuration, focus detection by the phasedetection method can be performed on the basis of an output signal fromthe image sensor 12 and any of the first through third image signalprocessing can be performed.(4) In the digital camera 1 in (2) or (3) above, the body control unit14 adds output signals from four adjacent R pixels in a two-by-twomatrix, adds output signals from four adjacent B pixels in a two-by-twomatrix, and adds output signals from four adjacent G pixels in atwo-by-two matrix, thereby forming an image signal of a Bayerarrangement (that is, performing the second image signal generationprocessing). With this configuration, optimal image signals can begenerated regardless of the incident angle of light into the microlenses40. Furthermore, a conventional image processing engine that performscolor interpolation of a Bayer arrangement can be used in the colorinterpolation processing.(5) In the digital cameras 1 in (1) through (3), the body control unit14 obtains three color signals at each position of microlens 40 on thebasis of output signals from the R, B and G pixels arranged behind eachof the microlenses 40 (that is, executes the first image signalgeneration processing). With this configuration, color image signals canbe obtained without performing color interpolation processing.(6) In the digital cameras 1 in (1) through (3), the body control unit14 executes color interpolation processing for generating signals fortwo other spectral components at each position of R, B and G pixels toobtain three color signals and generates a luminance signal and colordifference signals on the basis of the three color signals (that is,executes the third image signal generation processing). With thisconfiguration, image signals having high resolutions can be obtained.(7) In the digital cameras in (1) through (6) above, the body controlunit 14 detects the focusing condition of the interchangeable lens 2 onthe basis of outputs signals from a pair of pixels among the pixelgroup, the pair of pixels having substantially the same spectralsensitivities and having different positions with respect to themicrolens 40. With this configuration, the focusing condition can bedetected by a phase detection method appropriately based on the outputsignals from the image sensor 12.(8) In the digital camera 1 in (3) above, the body control unit 14detects the focusing condition of the interchangeable lens 2 in thehorizontal direction on the basis of output signals from G pixelsincluded in the first set P1 and the second set P2, respectively. Withthis configuration, the focusing condition can be detected in thehorizontal direction of the image sensor 12 by the phase detectionmethod appropriately based on the output signals from the image sensor12.(9) In the digital camera 1 in (3) above, the body control unit 14detects the focusing condition of the interchangeable lens 2 in thevertical direction on the basis of output signals from G pixels includedin the second set P2 and the fourth set P4, respectively. With thisconfiguration, the focusing condition can be detected in the verticaldirection of the image sensor 12 by the phase detection methodappropriately based on the output signals from the image sensor 12.(10) In the digital camera 1 in (3) above, the body control unit 14detects the focusing condition of the interchangeable lens 2 in adirection oblique to the horizontal direction on the basis of outputsignals from G pixels included in the second set P2 and the third setP3, respectively. With this configuration, the focusing condition can beappropriately detected by the phase detection method in an obliquedirection of the image sensor 12 on the basis of output signals from theimage sensor 12.

Variation Example 1

In the embodiment described above, the focus detection processing isexecuted by using output signals from G pixels. However, the focusdetection processing may also be performed by using output signals fromR pixels or B pixels.

The body control unit 14 according to Variation Example 1 is configuredto obtain AWB (auto white balance) evaluation values by using outputsignals from the image sensor 12. AWB evaluation values are cumulativevalues of output signals for each of R, G or B pixel. When thecumulative value for G pixel is low, it may be possible that outputsignals from G pixels cannot afford appropriate calculation of defocusamounts. Then, the body control unit 14 according to Variation Example 1executes, when the cumulative value for G pixel is equal to or lowerthan a predetermined threshold value, the above-mentioned focusdetection processing by using either one of R pixel and B pixel that hasa greater cumulative value than the rest. With this configuration,appropriate focus detection processing can be executed even whenshooting a subject having a small amount of G components.

Variation Example 2

In the embodiment described above, out of the first through third imagesignal generation processing, the processing that is indicated byinitial setting is used to generate image signals for recording.However, the present invention is not limited thereto.

For instance, the body control unit 14 according to Variation Example 2,when a through image is to be displayed, selects the first image signalgeneration processing in which image signals can be generated withoutexecuting color interpolation processing and generates image signals byusing the selected first image signal generation processing. On theother hand, for the images for recording, the third image signalgeneration processing capable of generating image signals having highresolutions is selected and image signals are generated by using theselected third image signal generation processing. As described above,the body control unit 14 according to Variation Example 2 is configuredto select, upon image signal generation, any of the first, second andthird image signal generation processing. With this configuration, imagesignal generation processing that is suitable for uses of images to begenerated can be selected. For instance, the first image signalgeneration processing which does not require any color interpolationprocessing is selected in a scene where it is desired to display imageson a real-time basis, whereas the third image signal generationprocessing is selected in a scene where it is desired to record imageswith high image quality.

In addition, the body control unit 14 may be configured to generateimage signals by the first or second image signal generation processingfor video images or by the third image signal generation processing forstill images.

In addition, the body control unit 14 may be configured to generateimage signals by using, for instance, both the first image signalgeneration processing and the second image signal generation processing.In this case, the body control unit 14 controls, for instance, both theimage generated by the first image signal generation processing and theimage generated by the second image signal generation processing to bedisplayed on a display device (not shown) on the rear side. The bodycontrol unit 14 records one of the two displayed images, which one isselected by the user through the operating member 18 into the memorycard 20.

Variation Example 3

In the embodiment described above, the configuration is adopted in whichthe defocus amount in the horizontal direction is obtained on the basisof output signal from the pixel row 90 constituted by the G pixel (G-b)included in the first set P1 and the G pixel (G-a) included in thesecond set P2. However, the present invention is not limited thereto. Aconfiguration may be adopted in which the defocus amount in thehorizontal direction is obtained on the basis of output signals from apixel row constituted by the G pixel (G-d) included in the third set P3and the G pixel (G-c) included in the fourth set P4. Alternatively, thedefocus amount in the horizontal direction may be obtained on the basisof both the above pixel row and the pixel row 90.

In addition, in the embodiment described above, the configuration isadopted in which the defocus amount in the vertical direction isobtained on the basis of output signals from the pixel row 120constituted by the G pixel (G-a) included in the second set P2 and the Gpixel (G-c) included in the fourth set P4. However, the presentinvention is not limited thereto. A configuration may be adopted inwhich the defocus amount in the vertical direction is obtained on thebasis of output signals from a pixel row constituted by the G pixel(G-b) included in the first set P1 and the G pixel (G-d) included in thethird set P3. Alternatively, the defocus amount in the horizontaldirection may be obtained on the basis of both the above pixel row andthe pixel row 120.

In the embodiment described above, the configuration is adopted in theoblique direction is obtained which the defocus amount in the verticaldirection is obtained on the basis of output signals from the pixel row150 constituted by the G pixels (G-a) and (G-d) included in the secondset P2 and the G pixels (G-a) and (G-d) included in the third set P3.However, the present invention is not limited thereto. A configurationmay be adopted in which the defocus amount in the oblique direction isobtained on the basis of the pixel row constituted by the G pixels (G-b)and (G-c) included in the first set P1 and the G pixels (G-b) and (G-c)included in the fourth set P4. Alternatively, the defocus amount in theoblique direction may be obtained on the basis of both the above pixelrow and the pixel row 150.

Variation Example 4

In the embodiment described above, explanation has been made on the casein which color filters of primary color system (RGB) is used in theimage sensor 12. However, color filters of a complementary color systemmay also be used.

Variation Example 5

In the embodiment described above, the present invention is applied tothe digital camera 1 having a configuration such that theinterchangeable lens 2 is mounted to the camera body 3. However, thepresent invention is not limited thereto. For instance, the presentinvention may also be applied to a lens-integrated digital camera.

Variation Example 6

In the third image signal generation processing in the embodimentdescribed above, it is configured such that at a position of anyparticular one of R pixels and B pixels, output signals from four Gpixels (G1 through G4) that located at positions nearest to theparticular pixel are used (cf., FIG. 12). However, the present inventionis not limited thereto. For instance, as shown in FIG. 18( a), aconfiguration may be adopted in which use is made of output signals fromfour pixels (G5 through G8) that are positioned in the horizontaldirection or in the vertical direction with respect to an interpolationtarget pixel and that are near the interpolation target pixel. In thiscase, the body control unit 14 defines (aG5+bG6+cG7+dG8)/4 to be a Gimage signal of the interpolation target pixel. It is to be noted that athrough d are coefficients that depend on respective distances of thenear G pixels from the interpolation target pixel. The larger thedistance from the interpolation target pixel, the larger the coefficientis set. In the case shown in FIG. 18( a), it is set a=b>c=d.

In addition, as shown in FIG. 18( b), a configuration may be adopted inwhich use is made of output signals from four G pixels (G9 through G12)adjacent via one intervening pixel in the horizontal direction or in thevertical direction to an interpolation target pixel. In this case, thebody control unit 14 defines (G9+G10+G11+G12)/4 to be a G image signalof the interpolation target pixel since the four G pixels have the samedistance from the interpolation target pixel. With this configuration,calculations weighted with coefficients can be omitted, so thatcalculation is made easier accordingly.

In, addition, the interpolation processing shown in FIG. 12 and theinterpolation processing shown in FIGS. 18( a) and (b) may be used incombination or one of them may be selected depending on the amount ofcalculation and/or accuracy of interpolation.

Variation Example 7

In the embodiment described above, the image sensor 12 may be a backsideillumination (BSI: Backside Illumination) type one. The configuration ofthe image sensor 12 in this case is described as follows.

FIG. 19 presents a diagram illustrating by an example a circuitconfiguration of the image sensor 12 according to Variation Example 7.In the image sensor 12 according to Variation Example 7, the circuitryis made up by grouping adjacent four pixels in a two-by-two matrixhaving the same color as one set. In FIG. 19, the circuit configurationof one set is exemplified.

Four photodiodes PD (PD1 through PD4) that correspond to four pixels,respectively, are connected to sources of transfer transistors TX (TX1through TX4) that correspond to the photodiodes PD (PD1 through PD4),respectively. Gates of the transfer transistors TX1 through TX4 areconnected to control lines Cn1 through Cn4, respectively, to whichtransfer pulse signals for turning on/off the transfer transistors TX1through TX4 are supplied. Drains of the transfer transistors TX1 throughTX4 are commonly connected to the source of a reset transistor RT. Thegate of the reset transistor RT is connected to control line Cn5, towhich a reset pulse signal for turning on/off the reset transistor RT isprovided. So-called floating diffusion FD between each drain of thetransfer transistors TX1 through TX4 and the source of the resettransistor RT is connected to the gate of a source follower amplifiertransistor SF. The drain of the source follower amplifier transistor isconnected to the source of a select transistor S. The gate of the selecttransistor S is connected to the control line Cn6, to which selectionpulse signals for turning on/off the select transistor S are provided.The drain of the select transistor S is connected to an output line Out.

The four photodiodes PD1 through PD4 perform photoelectric conversion oflight that has transmitted through each of color filters provided incorrespondence to the respective photodiodes to generate signal charges.The signal charges generated by the four photodiodes PD 1 through PD4are transferred to the floating diffusion FD through the transfertransistors TX1 through TX4. Since the floating diffusion FD isconnected to the gate of the source follower amplifier transistor SF, ifthe select transistor S is turned on, a signal corresponding to thepotential of the floating diffusion FD, after amplification by thefollower amplifier transistor SF, is output to the output line Out viathe select transistor S. The reset transistor RT discharges signalcharge of the floating diffusion FD to reset the floating diffusion FD.

It is to be noted that in the four photodiodes PD1 through PD4, thefloating diffusion FD, the source follower amplifier transistor SF, theselect transistor S, and the reset transistor RT are shared in common.Therefore, the image sensor 12 can output a signal for each photodiodePD or a signal obtained by adding signals from the four photodiodes PD1through PD4.

FIGS. 20( a) and (b) present diagrams illustrating by an example thearrangement of the circuitry described above in the image sensor 12. InFIG. 20( a), 8×8 pixels, which are extracted as representatives, areillustrated. In addition, FIG. 20( b) presents an enlarged diagramshowing a part of the 8×8 pixels that corresponds to one microlens 40(corresponding to 2×2 pixels) in FIG. 20( a). As shown in FIG. 20, thefloating diffusion FD, the source follower amplifier transistor SF, theselect transistor S, and the reset transistor RT are shared, not byadjacent four pixels in a two-by-two matrix behind one microlens 40, butby adjacent four pixels in a two-by-two matrix having the same color.Therefore, it is possible to read out from the image sensor 12 a signalpixel after pixel or a signal obtained by adding signals from theadjacent four pixels in a two-by-two matrix having the same color.Accordingly, when RGB signals for the respective microlenses 40 are readout in the first image signal generation processing described later, itis only necessary to read out a signal pixel by pixel. In addition, inthe second image signal generation processing, when the adjacent fourpixels in a two-by-two matrix having the same color are treated as onepixel, it is only necessary to read out a signal obtained by adding theoutput signals from the adjacent four pixels in a two-by-two matrixhaving the same color from the image sensor 12.

In addition, in the case of the image sensor 12 of the backsideillumination type, a wiring layer is provided on the surface side of asubstrate and a photodiode PD is provided on the backside of thesubstrate, which is opposite to the wiring layer, and light is incidentfrom the backside of the substrate. Therefore, in FIG. 20, themicrolenses 40 and RGB color filters as well as circuit layout (wiring)are shown so that all of them can be seen. However, in an actualconfiguration, if the image sensor 12 is seen from the incident plane oflight (backside of the substrate) as shown in FIG. 21( a), themicrolenses and the corresponding RGB color filters are seen but thecircuitry (wiring) is not seen, whereas if the image sensor 12 is seenfrom the side opposite to the incident plane of light (surface side ofthe substrate) as shown in FIG. 21( b), the circuitry (wiring) is seenbut the microlenses 40 and the corresponding color filters are not seen.In the case of the image sensor 12 of the backside illumination type,the aperture of the photodiode PD is broader than the aperture of thephotodiode in the conventional image sensor of the surface sideillumination type, so that loss of light can be decreased to enableimages to be captured with high sensitivity and at the same time, theaccuracy of the focus detection processing described later can beincreased.

In addition, the image sensor (image sensor chip) 12 according toVariation Example 7 may have a stacked structure in which it isconnected to a signal processing (DSP) chip 70 via a joint member 60(for instance, a micro bump or the like). FIG. 22 is a schematic diagramillustrating connection between the image sensor 12 and the signalprocessing chip 70. The image sensor 12 and the signal processing chip70 are stacked one above the other and connected with each other througha number of joint members 60. The joint member 60 is provided for eachset of adjacent four pixels in a two-by-two matrix having the samecolor. It is to be noted that for convenience of explanation, only thepart corresponding to one joint member 60 is illustrated.

The signal processing chip 70 is a chip that performs signal processingupon receipt of an output from the image sensor 12 and includes, forinstance, a circuit 71 that performs correlated double sampling (CDS)and analog/digital (A/D) conversion. It is to be noted that the signalprocessing chip 70 may be provided with a memory and a calculationcircuit.

Signals outputted from the photodiodes PD1 through PD4 of the imagesensor 12 are inputted in the circuit 71 of the signal processing chip70 via the joint member 60 and the circuit 71 performs correlated doublesampling (CDS) and analog/digital (A/D) conversion for the signals.

Variation Example 8

The image sensor 12 according to the embodiment described above may beconstituted by an image sensor of the surface side illumination type.

Variation Example 9

The image sensor (image sensor chip) 12 according to the embodimentdescribed above may be formed as one chip together with the signalprocessing chip 70.

The above description is by way of example and the present invention isnot limited to the embodiment described above. In addition, theembodiment described above may be combined with the configuration(s) ofone or more of the variation examples in any desired manner.

The disclosure of the following priority application is hereinincorporated by reference: Japanese Patent Application No. 2012-081167(filed on Mar. 30, 2012).

1. An image sensor comprising: a plurality of microlenses arranged in atwo-dimensional pattern; and a plurality of pixels that are provided incorrespondence to each of the microlenses and receive lights ofdifferent color components, respectively, wherein pixels that areprovided at adjacent microlenses among the microlenses and that receivelights of same color components, are adjacently arranged.
 2. An imagesensor according to claim 1, wherein signals from the pixels are usedboth for focus detection and for generation of image data.
 3. An imagesensor comprising: a plurality of microlenses arranged in atwo-dimensional pattern; and a plurality of pixels that are arranged incorrespondence to each of the microlenses and receive lights havingdifferent color components, respectively, wherein signals from thepixels are used both for focus detection and for generation of imagedata.
 4. A photographing method comprising: a focus detection step forperforming focus detection, by using signals from particular pixels inan image sensor including a plurality of microlenses arranged in atwo-dimensional pattern and a plurality of pixels that are arranged incorrespondence to each of the microlenses and receive lights havingdifferent color components, the particular pixels receiving lights ofsame color components from different microlenses, respectively; and ashooting step for generating a capture image by using signals fromplurality of pixels provided in correspondence to each of themicrolenses, the plurality of pixels receiving lights of different colorcomponents.
 5. A photographing method according to claim 4, wherein inthe image sensor, pixels are provided at adjacent microlenses among themicrolenses and that receive lights of same color components areadjacently arranged.
 6. An image-capturing device comprising: an imagesensor that captures an image of a subject with light fluxes from thesubject that have passed through an imaging optical system; an imagegeneration unit that generates an image signal based upon an outputsignal from the image sensor; and a focus detection unit that detects afocusing condition of the imaging optical system by a phase detectionmethod based upon an output signal from the image sensor, wherein theimage sensor includes a pixel group and a microlens group arranged so asto guide the light fluxes from the subject to the pixel group, the pixelgroup includes first, second and third pixels having first, second andthird spectral sensitivities, respectively, differing from each other,and being arranged in a two-dimensional pattern, with one of the firstpixels, one of the second pixels and two of the third pixels beingarranged in a two-by-two matrix behind each microlens in the microlensgroup, and the four pixels receive four light fluxes, respectively, thatpass through four pupil areas, respectively, of an exit pupil of theimaging optical system, the first, second, and third pixels are arrangedsuch that pixels having substantially same spectral sensitivities areadjacently arranged in a two-by-two matrix and four pixels adjacent tothe two-by-two matrix are arranged behind four different microlenses inthe microlens group, respectively, and at different positions withrespect to the microlenses, the image generation unit generates theimage signal based upon output signals from the first, second and thirdpixels, and the focus detection unit detects the focusing conditionbased upon an output signal from at least one of the first, second, andthird pixels.
 7. An image-capturing device according to claim 6, whereinthe first pixel has a red color filter, the second pixel has a bluecolor filter, and the third pixel has a green color filter.
 8. Animage-capturing device according to claim 7, wherein the pixel groupincludes an array of a plurality of sets of pixels arranged in atwo-dimensional pattern, each of the plurality of sets of pixels havingfour pixels arranged in a two-by-two matrix behind any particular one ofthe microlenses and the sets include first through fourth sets havingdifferent arrangements of pixels, in the first set, the first pixel andthe third pixel are adjacently arranged in a predetermined arraydirection and the third pixel and the second pixel are arranged adjacentto the first pixel and the third pixel, respectively, in a directionperpendicular to the predetermined array direction, in the second set,the third pixel and the first pixel are adjacently arranged in thepredetermined array direction and the second pixel and the third pixelare arranged adjacent to the third pixel and the first pixel,respectively, in the direction perpendicular to the predetermined arraydirection, in the third set, the third pixel and the second pixel areadjacently arranged in the predetermined array direction and the firstpixel and the third pixel are arranged adjacent to the third pixel andthe second pixel, respectively, in the direction perpendicular to thepredetermined array direction, in the fourth set, the second pixel andthe third pixel are adjacently arranged in the predetermined arraydirection and the third pixel and the first pixel are arranged adjacentto the second pixel and the third pixel, respectively, in the directionperpendicular to the predetermined array direction, the first set andthe second set are adjacent to each other in the predetermined arraydirection and alternately arranged in a repeated manner in thepredetermined array direction, the third set and the fourth set areadjacent to each other in the predetermined array direction andalternately arranged in a repeated manner in the predetermined arraydirection, and a first row formed by the first set and the second setand a second row formed by the third set and the fourth set are adjacentto each other in the direction perpendicular to the predetermined arraydirection and alternately arranged in a repeated manner in the directionperpendicular to the predetermined array direction.
 9. Animage-capturing device according to claim 7, wherein the imagegeneration unit adds output signals from four of the first pixels thatare adjacent to each other in a form of a two-by-two matrix, adds outputsignals from four of the second pixels that are adjacent to each otherin a form of a two-by-two matrix, and adds output signals from four ofthe third pixels that are adjacent to each other in a form of atwo-by-two matrix to generate an image signal of a Bayer arrangement.10. An image-capturing device according to claim 6, wherein the imagegeneration unit obtains three color signals at a position correspondingto each microlens based upon output signals from the first, second andthird pixels arranged behind each microlens.
 11. An image-capturingdevice according to claim 6, wherein the image generation unit performs,at respective positions of the first through third pixels, colorinterpolation processing for generating signals of other two spectralcomponents to obtain three color signals and generates a luminancesignal and color difference signals based on the thus obtained threecolor signals.
 12. An image-capturing device according to claim 6,wherein the focus detection unit detects the focusing condition of theimaging optical system based upon output signals from a pair of pixelshaving substantially same spectral sensitivities and located atpositions differing from each other with respect to the microlens, outof the pixel group.
 13. An image-capturing device according to claim 8,wherein the focus detection unit detects the focusing condition of theimaging optical system in the predetermined array direction based uponoutput signals from at least one plurality of the third pixels of aplurality of the third pixels contained in the first set and the secondset, respectively, and a plurality of the third pixels contained in thethird set and the fourth set, respectively.
 14. An image-capturingdevice according to claim 8, wherein the focus detection unit detectsthe focusing condition of the imaging optical system in the directionperpendicular to the predetermined array direction based upon outputsignals from at least one plurality of the third pixels of a pluralityof the third pixels contained in the first set and the third set,respectively, and a plurality of the third pixels contained in thesecond set and the fourth set, respectively.
 15. An image-capturingdevice according to claim 8, wherein the focus detection unit detectsthe focusing condition of the imaging optical system in a directionoblique to the predetermined array direction based upon output signalsfrom at least one plurality of the third pixels of a plurality of thethird pixels contained in the first set and the fourth set,respectively, and a plurality of the third pixels contained in thesecond set and the third set, respectively.
 16. An image sensorcomprising: a pixel group; and a microlens group arranged so as to guidesubject light fluxes to the pixel group, wherein the pixel groupincludes first, second and third pixels having first, second and thirdspectral sensitivities, respectively, differing from each other, andbeing arranged in a two-dimensional pattern, with one of the firstpixels, one of the second pixels and two of the third pixels beingarranged in a two-by-two matrix behind each microlens in the microlensgroup, and the four pixels receive four light fluxes, respectively, thatpass through four pupil areas, respectively, of an exit pupil of animaging optical system, the first, second, and third pixels are arrangedsuch that pixels having substantially same spectral sensitivities areadjacently arranged in a two-by-two matrix, respectively, and fourpixels adjacent to the two-by-two matrix are arranged behind fourdifferent microlenses of the microlens group, respectively, and atdifferent positions with respect to the microlenses.